ROPDetect : Detection of Code Reuse Attacks

Software exploitation, as used by malware and other kinds of attacks, require the attacker to take control of code execution. Historically, this involves injecting code into memory and using a software vulnerability to execute it. This works because both ARM and x86 uses a modified Harvard architecture which allows code and data memory to be shared. ARMv6 introduced the “execute never”[1] feature and Intel introduced the “execute disable” feature with their “Prescott” processors[3]. Both of these implementations ensure that memory pages are never mapped as both writable and executable (unless specified explicitly by the OS). This mitigates code injection attacks that relies on redirecting execution to attacker-controlled code stored in data memory. In response to this, attackers rely on “code reuse” or “return orientated programming” (ROP). The idea behind ROP is that the attacker cannot map her own code into the target’s executable memory, but she can “reuse” the code already in executable memory as well as control the target’s program stack (through some vulnerability). The program stack is typically used to store (along with other data), return pointers when a function call is made. The return pointer allows the program to resume execution at the caller after a function call returns. When the attacker overwrites the return pointer, she can redirect control flow to anywhere in executable memory. If the attacker finds useful “gadgets”, or instructions that perform a single useful operation (such as a memory load or store) and then jumps to the next return pointer in the stack, she can inject a large number of return pointers into the stack and control execution that way. Even though ROP attacks are very powerful, most attackers only use it as the first stage of a multi-stage attack where ROP is used to bypass operating system restrictions in order to escalate control of the compromised process to control of the entire system. If we can differentiate ROP execution with normal code execution, then we can terminate a process before the control is escalated and stop the attack. Heuristically, we can see that ROP execution is different because it uses a large number of “return” instructions while normal (optimized) code does not perform as many returns in the same sort period of time. Additionally, in normal code, “return” instructions are often matched with “call” instructions. Finally, processors optimized to run normal code will likely perform better with normal execution than ROP execution (which for example, makes branch prediction hard). However, even in normal execution, there are situations such as tail recursion in a tight function that may make classification using just heuristics difficult. We choose to implement unsupervised machine learning to solve the classification problem.